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ABSTRACT 



Horizontal teacher evaluation increases understanding of 
teaching through the critical examination of educational goals and classroom 
practices. This examination includes teachers' classroom observations of one 
another coupled with in-depth teacher conversations or dialogue about 
teaching aims and practices. A qualitative case study of a horizontal teacher 
evaluation program in an urban school district is described. Data collection 
and analysis in the study generally follow grounded theory in qualitative 
research. Data came from classroom observations, formal and informal teacher 
and principal interviews, questionnaires, and documents that include program 
meeting notes, dialogue transcripts, and teachers' action plans. There were 
15 teachers involved in horizontal evaluation in 2 elementary schools. 

Results of the study support the claim that formative teacher evaluation can 
result in changes in teachers. Every teacher in the horizontal evaluation 
program made changes related to teaching. The program appeared to be 
successful in meeting teacher needs, as demonstrated by the positive 
attitudes of teachers toward the program and the changes that occurred for 
all participants. The results also support the idea that teachers want 
opportunities for evaluation and development when they feel that these 
activities really help them and are not forced on them. An appendix describes 
the study coding scheme. (Contains 36 references.) (SLD) 
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Background 

In recent years, teacher evaluation has become an important focus in efforts to improve 
education. Numerous teacher evaluation systems have been developed in an effort to identify and 
dismiss poor teachers and help adequate teachers improve their practice. Most public schools 
have some type of teacher evaluation system in place. In addition, over 40 states have laws or 
regulations mandating teacher evaluation (Valentine, 1992). But in spite of the widespread use of 
teacher evaluation, the impact of teacher evaluation on teacher and school improvement has been 
disappointing (Bosetti, 1994; Duke, 1993; Stiggins & Duke, 1988). Many researchers believe 
that the impact of teacher evaluation on educational improvement has been minimized because of 
the prevalent focus on teacher accountability (e.g., Bosetti, 1996; Gitlin & Smyth, 1989; Haefele, 
1992; McLaughlin, 1990). 

Researchers who hold this view support approaches to teacher evaluation that emphasize 
teacher development. They argue that development focused teacher evaluations have the 
potential to help nearly all teachers improve their practice, while accountability focused 
evaluations are aimed at only the small percentage of teachers who's job status is in question. In 
addition, development focused teacher evaluations provide specific information on how teachers 
can improve instruction rather than just information on whether a teacher is deficient or adequate. 

One of the recent efforts to move from accountability focused teacher evaluation to 
evaluations that focus on teacher development is horizontal teacher evaluation (see, Gitlin & 
Goldstein, 1987). Horizontal teacher evaluation focuses solely on teacher development; no part 
of the evaluation process or data gathered is used for teacher accountability. The purpose of 
horizontal evaluation is to increase understanding of teaching through the critical examination of 
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educational goals and classroom practices. This examination includes teachers’ classroom 
observations of one another coupled with in-depth teacher conversations or dialogue about 
teaching aims and practices. 

Program Description 

Horizontal teacher evaluation begins with a group meeting of all participants. Subsequent 
group meetings are held about every month throughout the course of the program. The purpose 
of these meetings is to train teachers in the horizontal evaluation approach and discuss 
instructional topics that are of interest to teachers. The first group meeting is also used to help 
teachers get to know one another and choose a partner to work with in the program. It is usually 
best for teachers to work in pairs, but it is also possible for teachers to work together in groups, of 
three or more. 

Once teachers are familiar with the horizontal evaluation approach and have chosen 
partners, they are given the opportunity to observe each other in their respective classrooms. 

Each partner should observe and be observed. These observations are followed by horizontal 
dialogue between teachers. This dialogue "does not pit one actor against another but rather 
enables participants to work together to understand the subject being discussed." (Gitlin, 1990, p. 
540). Repeated observations and dialogue continue throughout the duration of the horizontal 
evaluation program. The purpose of these observations and dialogue is to increase understanding 
of teaching through the critical examination of educational goals and classroom practices. 
Teachers look for the gap between what they desire to do in their teaching and their actual 
classroom practice. They also examine the reasons they have chosen certain goals and why they 
engage in certain classroom practices. 
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The examination of teachers’ goals and practices during teachers’ dialogue includes four 
major elements: communication analysis, historical perspective, challenge statements, and 
considering alternatives (Gitlin & Goldstein, 1987). Each of these elements is designed to help 
teachers talk about teaching in productive and insightful ways. 

Communication Analysis 

Communication analysis is the examination of words and sentences that appear in speech 
to uncover prejudgements and distortions in understanding. Based on Habermas's (1979) four 
validity claims present in every speech act, Gitlin and Goldstein (1987) outline elements of 
communication analysis in horizontal teacher evaluation. These include: 

• Comprehensibility — considering whether the teacher's statement is understandable 

• Truth — whether the statement is a true description of what happened 

• Sincerity — whether teachers are stating their actual perceptions and feelings 

• Appropriateness — whether the statement or labeling is morally justifiable 

Through this four pronged analysis of communication, both the teacher and evaluator are required 
to examine more closely their perceptions of teaching and understanding of classroom events. 
Historical Perspective 

Developing a historical perspective in horizontal teacher evaluation involves the analysis 
of current realities in relation to past events. Gaining a historical perspective helps teachers and 
evaluators understand the roots of present norms and practices and consider their value in 
establishing and reaching educational goals. A historical perspective may be achieved by writing 
teacher personal histories or discussing how teachers have come to teach as they do (i.e., the 
events, forces, ideologies, and thought patterns that have shaped their present practice). For 
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example, a teacher may have a strong commitment to having students write about their personal 
feelings and opinions on given subjects in class. In this case, developing a historical perspective 
may involve a discussion of past events in the teacher’s life that helped shape his/her commitment 
to students’ personal writings. Once these events are identified and described, they can be 
examined to help determine the value and appropriateness of students’ personal writings in the 
course. 

Challenge Statements 

Challenge statements are direct questions about the appropriateness of teachers' intentions 
and practices. Challenge statements may also question the evaluation process and intentions and 
practices of the evaluator. "The purpose of challenge statements is to push dialogue beyond the 
simple clarification of values and prejudgements to the point where issues of rightness can be 
raised in relation to a mutually recognized normative framework." (Gitlin & Goldstein, 1987, p. 
20). Teachers often assume goals and practices that are predetermined by others or developed by 
themselves without a critical analysis of the appropriateness of these goals and practices. 
Challenge statements help teachers and evaluators examine educational assumptions and the value 
of present educational aims and practices. 

Considering Alternatives 

Considering alternatives is used in horizontal evaluation to explore different educational 
approaches or methods. This exploration helps illuminate possible alternatives and evaluate their 
implications for teaching and learning. A critical assessment of the values and implications of 
different approaches or methods is an important part of considering alternatives. The assessment 
of values and implications helps teachers better understand the utility and appropriateness of 
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alternatives. In addition, this assessment helps provide teachers with an understanding of when 
and how to use the alternative approaches or methods in teaching practice. 

Methodology 

The present research is a qualitative case study of a horizontal teacher evaluation program 
operating in an urban school district. Data collection and analysis in the study generally follow 
grounded theory in qualitative research (see, Glaser & Strauss, 1967). Grounded theory is an 
approach to qualitative research in which theory is allowed to emerge from the continual 
interchange of systematic data analysis and theory building throughout the course of a study. As 
the study progresses, the most important factors related to the phenomenon of interest are 
identified and the research becomes more and more focused on these factors. Strauss and Corbin 
(1990) explain: "A grounded theory is one that is inductively derived from the study of the 
phenomenon it represents. That is, it is discovered, developed, and provisionally verified through 
systematic data collection and analysis of data pertaining to the phenomenon." (p. 23). 

This does not mean, however, that one begins research with no research questions in mind 
Research questions help guide the research. According to grounded theory, beginning research 
questions are broad enough to provide "the flexibility and freedom to explore a phenomenon in 
depth," yet, "not so open, of course, as to allow for the entire universe of possibilities." (Strauss 
& Corbin, 1990, p. 37-38). In the present study, the general questions that guided research were: 
(a) What changes occurred in teachers related to their classroom practice during the time they 
participated in horizontal evaluation? (b) Were these changes related to teachers’ participation in 
horizontal evaluation? 
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Participants 

The participants in this study were 15 teachers from two elementary schools. There were 
ten teachers from one school and five from the other. Teachers agreed to participate in the study 
as part of a horizontal teacher evaluation program in which they were enrolled for college credit. 
The teachers were all white middle class. One teacher was male and the rest were female. All 
teachers had at least a bachelors degree and three had masters degrees. The grades taught by 
these teachers ranged from 1st to 6th with a slightly larger proportion of teachers in the upper 
elementary grades. Teachers' prior classroom experience ranged from zero (a first year teacher) 
to 34 years. The students in these teachers' classrooms were predominantly white middle class in 
one school and white lower middle class in the other. 

Procedure 

Data for this study were collected during the 1994-95 school year (September through 
May). The most intensive data collection occurred during the first and last six weeks of this time 
period. The intensive data collection at the beginning of the school year helped establish base-line 
data on teachers’ thoughts and practices before the Horizontal Teacher Evaluation Program was 
implemented. The intensive data collection at the end of the school year provided data on 
teachers’ thoughts and practices at the completion of program. Additional data were gathered 
throughout the school year to provide an understanding of the horizontal teacher evaluation 
process and the development of teachers' thoughts and practices as the program progressed. 

This study utilizes seven data collection procedures. These procedures include classroom 
observations, formal teacher and principal interviews, informal teacher and principal interviews. 
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teacher and principal questionnaires, notes from program meetings with teachers, transcripts of 
dialogue between teachers, and teachers' action plans. 

Data Analysis 

Data analysis involved four general steps or activities. These included developing themes, 
identifying patterns of relationships, confirming patterns of relationships, and relating themes and 
patterns of relationships to the existing literature (see. Miles & Huberman, 1994; Strauss & 

Corbin, 1990). 

Themes were developed by coding and categorizing data. Data were broken down into 
specific incidents, statements, or ideas and given a conceptual name. These concepts were 
grouped into categories that represent general themes. Each concept was a subcategory of the 
general category or theme in which it was placed. In some cases, the examination of 
subcategories resulted in redefining or renaming themes. In other cases, subcategories were 
rearranged among existing themes or placed under newly created themes. A brief description of 
the coding scheme appears in Appendix A. 

After themes were clearly defined, they were closely examined to identify and verify 
possible relationships between themes. Possible relationships were checked by returning to the 
data and looking for evidence that either supported or refuted them. When the data supported a 
given relationship, this relationship became a kind of working hypothesis that helped guide further 
questioning, data collection, and analysis. In cases where the relationship was not confirmed, 
further questioning and reshaping of the proposed relationship was made until the relationship was 
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At the end of data analysis, the themes and relationships between themes were compared 
to the existing literature to see what other researchers had found regarding the same or similar 
issues. This comparison was done for a number of reasons. First, it allowed for study findings to 
be put into the larger context and explained in relation to larger issues (see. Miles & Huberman, 
1994). Second, it helped researchers look at the data in different ways and from different 
perspectives (see, Strauss & Corbin, 1990). Third, it helped explain why certain patterns 
occurred in the present study (see. Miles & Huberman, 1994). 

Results 

Information on changes in teachers during the course of the Horizontal Teacher 
Evaluation Program is organized into three areas. These areas are teachers' goals, teachers' 
thoughts and attitudes, and teachers' actions. 

Teachers' Goals 

All 15 teachers in the Horizontal Teacher Evaluation Program changed or modified their 
goals during the course of the program. For three of the 15 teachers, the changes were small. 
They modified their goals and direction but did not depart substantially from the goals they held at 
the beginning of the school year. Ten teachers experienced considerable changes in their goals 
and direction. For these teachers, approximately one third to two thirds of their goals were 
different at the end of the program than they were at the beginning. In addition, their goals at the 
end of the program represented at least one major shift in direction or focus in their teaching. 

Two teachers experienced an almost complete change in their goals and direction. For both of 
these teachers, only one goal they stated at the beginning of the program was expressed at the end 
of the program. 




10 



Horizontal Teacher Evaluation 10 



Teachers' goals changed in a number of ways. The two teachers that experienced the 
greatest change in their goals generally moved from more teacher centered goals to goals that 
were more student centered. For example, these teachers’ goals at the beginning of the program 
included (a) have a purpose in what you do, (b) have respect between teacher and students, (c) 
keep students progressing, and (d) be a role model. At the end of the program, their goals 
included (a) be more interested in listening to students, (b) raise students’ self-esteem, (c) be more 
sensitive to students’ different levels of learning, and (d) meet individual students’ needs by 
identifying and teaching to their learning styles. 

The ten teachers who experienced considerable change in their goals had changes in 
different directions. Four of the 10 teachers changed their goals to include more emphasis on 
students' needs and on the teacher's role as a facilitator of student learning. Two teachers 
changed their goals to focus more on relationships in teaching and learning. They emphasized 
relationships between the teacher and students and between students. One teacher's goals 
changed to emphasize motivating students to learn. Another teacher changed from goals centered 
on the teacher-student relationship to goals that focused on students' self-worth and positive 
experiences in the classroom. One teacher moved from more student focused goals, in which the 
teacher was a facilitator, to goals that focused on teaching learning skills. Another teacher 
changed her goals to place more emphasis on helping students contribute to society. 

Although it is difficult to identify the exact relationship between the Horizontal Teacher 
Evaluation Program and teachers' goals, there is evidence that the program influenced these goals. 
First, most of the teachers changed their goals in the general direction anticipated. Teachers in 
the program were expected to change in ways that encouraged more student participation, self- 
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direction, collaboration, reflection, and higher-order thinking. Of the 12 teachers who made 
considerable or major changes in their goals, nine teachers changed in ways that support one or 
more of the anticipated outcomes. Two teachers' goals changed in ways that were neither more 
toward nor more away from the anticipated outcomes. One of the 12 teachers seemed to change 
her goals away from the anticipated outcomes. 

Second, the program seemed to help teachers analyze their goals and understand the 
purposes behind their actions. One teacher, after analyzing her goals, said she found some of her 
goals "were kind of stupid." Another teacher described the program's influence on understanding 
her purposes in teaching: 

Definitely it [the program] has had a big impact, I think in a positive way, of helping me 
start to do more reflecting and to think about what I'm doing and why I'm doing it. Am I 
doing it for me or am I doing it for the benefit of the kids? ... A lot of times you just do 
things because you've always done them that way .... Some things you've learned from 
other teachers and they're not necessarily the right way or a good way for you or your 
class. So this made me kind of start to think about it and why I do certain things. 

Third, teachers reported that their participation in the Horizontal Teacher Evaluation 

Program influenced their goals. For example: One teacher said that before the program started, 

she had the goal that "everyone will have homework and everyone is going to do it." She told 

how participation in the program had helped her understand that some students' home situations 

prevented them from doing the homework she required. She changed her goal to allow for some 

students to do most or all of their work at school. Another teacher told how she and other 

teachers had developed goals for their participation in the program the coming year. Their group 

was going to focus on understanding multiple intelligences and using constructive assignments. 

Through participation in the program, many teachers developed the goal to meet with students' 
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parents in their homes at the beginning of the following school year. This goal was shared by 
teachers at both schools. 

Some teachers reported that, in addition to facilitating the change or development of 
goals, the Horizontal Teacher Evaluation Program influenced the focus or realization of their 
goals. For example: One teacher said the program helped her focus on goals she already had but 
usually let slide. Another teacher reported that the program helped her focus on a previously 
established goal of improving classroom management. The program also helped this teacher 
change from her focus on lesson objectives to focus on broader classroom goals. A third teacher 
told how the program helped her quickly achieve her goals and move on to other goals. Her 
original goals were accomplished in the first two months of the program. This made it possible 
for her to establish and accomplish additional goals during the course of the program. 

Teachers' Thoughts and Attitudes 

Teachers' thoughts and attitudes is a broad category that includes teachers’ reflections on 
teaching, relationships and collaboration with other teachers, learning about teaching, acceptance 
of being observed while teaching, attitudes toward teacher evaluation, and attitudes toward the 
Horizontal Teacher Evaluation Program. 

Reflections on teaching . 

Probably the most noticeable effect of the Horizontal Teacher Evaluation Program on 
teachers was in their reflection on teaching. The program seemed to help increase reflection in all 
teachers that participated. This increased reflection was clearly seen in the dialogue sessions. As 
the program progressed, there was a distinct change in how teachers talked with one another 
about their teaching. Beginning dialogue sessions were characterized by the observing teacher 
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telling the observed teacher what he or she saw in the other's classroom. These comments were 
usually followed by brief comments by the observed teacher. By the end of the program, the 
dialogue sessions consisted of very brief comments by the observing teachers. Most of the talking 
was done by the teachers who were observed. These observed teachers' comments demonstrated 
in-depth reflection on the events that occurred in the classroom during the observation. They 
talked at length about their thoughts, actions, and feelings in relation to these events. They were 
able to understand and analyze their own teaching to a much greater extent than they did at the 
beginning of the program. 

The effects of the program on reflection were also seen in teachers' comments at the end 
of the program. The majority of teachers in the program mentioned how the program had 
affected their reflective thoughts. For example: One teacher told how sharing her thoughts with 
her partner in the program increased the effectiveness of her reflectivity. Another teacher said 
that the program helped her reflect on the growth she had made. One teacher explained how 
discussing her teaching with others forced her to analyze her teaching. Other teachers commented 
on how their reflection on teaching had increased as a result of the program. 

Relationships and collaboration with other teachers . 

Many teachers commented on the effects of the Horizontal Teacher Evaluation Program 
on their relationships and collaboration with other teachers. One teacher told of the change in the 
group of teachers she worked with in the program. She said, "We're more unified. We're more 
trusting. We're more helpful. Even though we were that before, it's a new level." Another 
teacher described the changes she and other teachers experienced. She said, "We've got so much 
more respect for each other and for our differences." She said she could now count on her 
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colleagues because she really knew them. She also found that her colleagues trusted and counted 
on her. In reference to her colleagues she said, 'Tve been absolutely floored when a couple of 
these people have come to me and said, ‘Will you help me with this?' or 'Can we do this 
together?"' She explained what a change this was from her relationship with these teachers before 
the program. She said, "Before it was like they were just kind of a stranger. They would kind of 
smile and say everything is fine. It was all surface. It was all a facade." 

Learning about teaching . 

Participation in the Horizontal Teacher Evaluation Program seemed to be an important 
tool in helping teachers learn more about teaching. Teachers reported that the program helped 
them gain new ideas about teaching and learn new teaching methods and techniques. Teachers 
said this learning came primarily from talking with others about their teaching and observing other 
teachers in their classrooms during the course of the program. 

Acceptance of being observed while teaching . 

The Horizontal Teacher Evaluation Program also affected teachers' thoughts and attitudes 
about people observing their teaching. This change was seen by both principals and teachers. 
Referring to his classroom observations of teachers, one principal said, "I felt like they [teachers] 
felt more at ease with me being there. . . . This year there was less stress about me coming in." 

He explained that when appointments for teacher evaluations fell through, he had some teachers 
tell him to just come in anytime to do the observation. The principal from the other school made 
similar comments. She experienced an exceptional openness of teachers to her classroom 
observations. This increased openness of teachers to their respective principals occurred even 
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though the principals continued to be the ones who evaluated teachers for accountability purposes 

in the district's summative teacher evaluation system. 

Many teachers commented that the program helped them feel more comfortable with other 

people in their room. One teacher said, "I'm a lot more comfortable having visitors. ... I feel 

comfortable with people in there and my kids feel comfortable. It's not weird to have a visitor 

come in." Another teacher said he no longer felt intimidated when other teachers came into his 

classroom. He explained that teachers who observed him were not looking for faults. Instead, 

they were looking for strengths and making suggestions. Another teacher described the effects of 

the program on her feelings about being observed: 

At this point after 34 years, suddenly I feel a little bit more in charge, that anybody can 
come into my room and that I pretty much have a general idea of what I'm doing. Not 
that everything is perfect! But, you know, that I shouldn't be threatened if somebody 
walks into my room to see what's going on. 

Attitudes toward teacher evaluation . 

The majority of teachers said that participation in the Horizontal Teacher Evaluation 
Program had affected their attitudes about teacher evaluation. They said they had gained new 
understanding of what teacher evaluation could be. They found that evaluation could be a tool 
for improvement rather than just a measure of competence. One teacher said that because of the 
program she was starting to see teacher evaluation as something to help her rather than a way to 
judge her. Another teacher said that she no longer tried to hide her weaknesses. Instead, she 
wanted to identify her weaknesses and work on them. Another teacher explained the change she 
had experienced in this way: 

Teacher evaluation before was the principal coming in and marking you down in areas he 
saw lacking. Teacher evaluation now is really helpful. . . . It's not putting on a show for 
somebody to come in and evaluate. It's an ongoing process that helps you reflect on your 
teaching, that helps you improve in areas where you think you're lacking. 




16 



Horizontal Teacher Evaluation 16 



Attitudes toward the Horizontal Teacher Evaluation Program . 

Teachers also changed their attitudes about the Horizontal Teacher Evaluation Program. 
The apprehension that many teachers felt at the beginning of the program disappeared as the 
program progressed. By the end of the school year, teachers unanimously supported the 
program. Every teacher wanted to continue the program in the coming year. They also hoped to 
get additional teachers to participate in the program with them. The only suggestions for 
improvement were made by two teachers who wanted to move faster in the program. One of 
these teachers wanted to understand the program and share more with other teachers sooner in 
the program. The other teacher wanted to more quickly and directly address and solve teaching 
problems. 

Teachers' Actions 

Analysis of the effects of the Horizontal Teacher Evaluation Program on teachers' actions 
yielded mixed results. There was little change seen in the specific teacher actions that were the 
focus of classroom observations. On the other hand, many teachers reported changes they had 
made as a result of the program. 

The specific teacher actions that were the focus of classroom observations remained 
consistent throughout the course of the program. There were no identifiable changes in the types 
of questions teachers asked students; teachers preference for individual, small group, or whole 
class activities; or the amount of time teachers spent presenting information to students. There 
was very little change in the types of activities teachers used in the classroom. No changes were 
seen in individual and small group activities. There was a slight increase toward more open, 
student directed activities when the whole class participated as a group. Classroom discipline was 
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the only area where there was a clear pattern of change. The number of teachers' actions directed 
toward classroom discipline and control decreased as the school year progressed. 

Many teachers reported that changes they made related to teaching were the result of 
participation in the Horizontal Teacher Evaluation Program. Some teachers made general 
statements in this regard. For example: One teacher said that because of the program her teaching 
and the teaching of those she worked with had improved. She said she was challenging herself 
and was not stuck anymore. A number of teachers gained new ideas from participating in the 
program which they applied in their classrooms. One teacher realized that the program helped her 
have more power in her classroom. She explained, "I thought I had a lot of power about what I 
did in the classroom. Now I realize where the power is coming from. It's from all of us meeting 
together." Other teachers commented on how participation in the program helped reinforce and 
solidify many of the good things they were already doing in the classroom. 

Some teachers' comments were more specific. One teacher told how participation in 
horizontal evaluation helped her analyze and evaluate an instructional program she had used in her 
classroom for years. This program had been a "sacred cow" for her, but her attitude about the 
program changed. She saw that much of the program was not what she wanted for her and her 
students. She stopped using most of the program. She kept only certain aspects of the program 
that she felt were most beneficial. 

One teacher described the impact of the Horizontal Teacher Evaluation Program on her 
organization of teaching materials. In discussing her program goal to become more organized, 
she said, "I pulled every single thing out. If it wasn't laminated, it got laminated and cut. If it 
wasn't organized, then I organized it into month folders." Throughout the year, she even made 
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double copies of all her student materials. In reference to these materials, she said, "I have every 
single thing for next year done." This organization seemed to have an effect on her in the 
classroom. She explained, "I've already seen a difference in my personality. I'm so calm. It's like 
I have no worries." The organization also helped her do more with her students and develop 
closer relationships with them. She said that her students had come to know her better as a 
person and that she was more of a friend to them. 

One teacher told how the Horizontal Teacher Evaluation Program helped her read, study, 
and focus more on her goal of classroom management. Another teacher said that her "tone" and 
"touch" in the classroom had changed. Other teachers described how the conversations between 
teachers had changed since the program began. Instead of complaining about problems, teachers 
were talking more about solutions. They were sharing ideas and strategies on how to solve 
problems. 

Discussion of Unexpected Outcomes 

There are two areas where unexpected outcomes of the Horizontal Teacher Evaluation 
Program occurred. These two areas are (a) the lack of change in teachers' actions observed in 
classroom observations and (b) one teacher who seemed to change her goals away from the 
anticipated program outcomes. 

Lack of Change in Teachers’ Observed Actions 

Probably the most important unexpected outcome of the program was the lack of change 
in teachers' actions during classroom observations. At first glance, this lack of change seems to 
contradict the changes that teachers reported were a result of the program. A closer look reveals 
that even though there was a lack of change in observed teacher actions during classroom 



Horizontal Teacher Evaluation 19 



observations, this is not necessarily at odds with the teachers' self-reports. The actions observed 
in classroom observations and the changes teachers reported were usually not in the same areas. 
For example, many of the changes teachers reported were in areas such as organization of 
teaching materials, programs they were using, record keeping, listening to students, interviewing 
students, conversations between teachers, and the amount of energy spent on goals. These areas 
are only indirectly related, if related at all, to the actions identified in classroom observations (i.e., 
types of questions teachers asked students; teachers preference for individual, small group, or 
whole class activities; amount of time teachers spent presenting information to students; amount 
of student directed activities; and number of teachers’ actions aimed at classroom discipline). 
Therefore, it was possible for teachers to make changes in aspects of their teaching that were not 
apparent in the classroom observations. 

Another reason teachers’ self-reported changes may not have been apparent in classroom 
observations is the time in took for teachers to implement change. Before the Horizontal Teacher 
Evaluation Program could elicit changes in teaching, teachers had to learn how to utilize the 
program, come to understand their own teaching, identify areas for change, and make specific 
plans to bring about the changes they desired. This process took time. Many teachers in the 
program said they were just beginning to understand and utilize the program by the end of the 
school year. One teacher explained: 

We're just barely starting. We are not doing real horizontal evaluation yet. I think its 
taken us a year to understand and feel secure and safe with each other. It takes a long 
time for people to feel safe and understand. 

Another teacher commented, "The only bad thing that I haven't liked is that I don't feel as a 
partnership we've really even touched the surface." Other teachers believed that many changes 
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could not be made in one year. They said that most changes wouldn't be visible until the coming 
school year. 

Since changes in teaching behaviors may be understood as changes in multiple cognitive 
skills applied simultaneously in real world settings, it is to be expected that teachers would go 
through stages in acquiring these skills (see, Anderson, 1982). The changes teachers reported 
may have coincided more with the covert stages of skill acquisition when overt changes in teacher 
behaviors were not yet apparent. Therefore, it is possible that it was premature to expect the 
anticipated program outcomes to be observed in classroom observations during the first year of 
the program. Changes in these outcomes may not become apparent until longer after teachers 
participated in horizontal evaluation. 

Change in Goals Awav from Anticipated Program Outcomes 

It is interesting that one teacher seemed to change her goals in a direction opposite to the 
anticipated program outcomes. This teacher changed her goals from more of a student centered 
focus where the teacher is a facilitator to an emphasis on teaching learning skills. The reason for 
change in this direction is unclear. It is possible that the changes were in response to the specific 
needs of the teacher and her class. This teacher expressed a need to focus more on classroom 
discipline and control. Her action plan and much of her focus in the Horizontal Teacher 
Evaluation Program was in this area. This need might have prompted her to move her goals away 
from a student centered approach where the teacher is a facilitator. She may have felt a need in 
her class to take more control and directly teach learning skills to her students. 
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Implications of Results 

The results of this study support the claim that formative teacher evaluation can result in 
changes in teachers. Every teacher in the Horizontal Teacher Evaluation Program made changes 
related to their teaching. This is in stark contrast to the lack of change seen in teachers as the 
result of summative teacher evaluations (see, Bosetti, 1994; Duke & Stiggins, 1986; Kauchak, 
Peterson, & Driscoll, 1985; McGreal, 1988; Wise & Darling-Hammond, 1985). It also contrasts 
the slow and difficult process of teacher change encountered by most teacher development 
programs (Guskey, 1986). Fullan and Stiegelbauer (1991) explain that most efforts aimed at 
teacher change fail because they do not adequately meet teachers' varying needs and concerns. 
The Horizontal Teacher Evaluation Program appeared to be successful in meeting teachers' 
various needs. This is demonstrated by the positive attitudes of all teachers toward the program 
and the changes that occurred in all teachers who participated. This universal change in teachers 
is important, especially when considering the various teachers involved. The program elicited 
change in teachers of different grade levels and varying levels of teaching experience. Indeed, the 
program seemed as effective for the first year teacher as it was for the teacher who had taught for 
34 years. This is exceptional when considering the very different formative evaluation needs of 
teachers at different stages of their professional development (see, Berliner, 1990; Costa, 
Garmston, & Lambert, 1988; Glickman, 1990). The ability of the program to meet these needs 
and elicit widespread change in teachers is very encouraging. It demonstrates that formative 
teacher evaluation programs can affect a large percentage (in this case 100%) of teachers 
involved. 
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The results of this study are also important because of the types of changes teachers 
experienced. Positive changes occurred in teachers' goals, actions, reflection, collaboration, 
knowledge about teaching, and attitudes about evaluation. Changes in teachers' goals and actions 
directly impact classroom practice. Positive changes in reflection, collaboration, knowledge about 
teaching, and attitudes about evaluation are also important elements in the improvement of 
teaching. This is seen in the literature in each of these areas. 

Reflection 

Researchers and practitioners emphasize the importance of teacher reflection in teacher 
improvement. For example: Evans (1995) describes the importance of critical teacher reflection 
in improving instruction and learning and tailoring instruction to students needs. Hellison and 
Templin (1991) believe that "reflective teaching offers the best opportunity for both preservice 
and in-service teachers to learn how to teach well." Richardson (1990) describes the importance 
of teacher reflection in teacher education and development. 

Collaboration 

Researchers also report positive benefits of teacher collaboration. Gay and Ross (1994) 
found that teachers working together in support of one another resulted in outstanding physical 
education programs. Adajian (1996) reports that a strong professional community among 
teachers is important in increasing teachers' knowledge, expertise, and motivation to improve. 
McAllister and Murphy (1990) suggest a link between teacher collaboration and improvement in 
students' grades. Hargreaves (1994) summarizes many benefits of collaboration including 
improved teaching and learning. 
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Knowledge About Teaching 

Teachers' knowledge about teaching is considered a fundamental component in the 
improvement of teaching. Borko and Putnam (1995) describe the importance of teachers' 
knowledge in teachers coming to understand and make decisions about new teaching practices 
and activities. This knowledge determines how new instructional tools are actually used in the 
classroom. Duckworth (interview, Meek, 1991) describes the importance of teachers' gaining 
understanding of teaching and learning from the learners' perspective. This understanding helps 
teachers meet students' instructional needs and provide the groundwork for curriculum 
development. Smylie (1995) emphasizes the importance of teachers gaining new knowledge to 
improve their teaching. He relates research on adult learning to the acquisition of new knowledge 
by teachers. In summarizing this research he states: "Learning occurs as individuals confront and 
alter taken-for-granted assumptions to reframe problem situations." (p. 94). This learning leads to 
the identification and analysis of problems and decisions to act or not act on a particular solution. 
This process for optimum adult learning described by Smylie is very similar to the way teachers 
acquired and used knowledge in the Horizontal Teacher Evaluation Program. 

Attitudes About Evaluation 

Teachers' attitudes about evaluation are also linked to teacher improvement. Overcoming 
negative attitudes about teacher evaluation is a major obstacle to the effectiveness of formative 
evaluation in improving teaching. Most current teacher evaluation approaches raise teachers' 
distrust of the evaluation process (Gitlin & Smyth, 1990). They encourage teachers to conceal 
and even deny weaknesses rather than identify weaknesses and work to improve (Haefele, 1992). 
Duke (1993) explains that even when growth oriented teacher evaluations are instituted, 
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professional growth may be hindered because of teacher disillusionment, distrust, and pessimism. 

In light of these concerns, the changes in teachers' attitudes toward evaluation while 
participating in the Horizontal Teacher Evaluation Program are very encouraging. Teachers 
changed from skepticism to actually wanting formative evaluation. Teachers started to see 
evaluation as a tool to help them improve rather than a way to judge them. Teachers in the 
Horizontal Teacher Evaluation Program frequently expressed how much they liked the program. 
They also expressed a desire to continue the program in the coming year. This desire was 
expressed by all teachers regardless of whether or not they would receive credit for their 
participation or funding to implement the program in the coming year. 

Conclusion 

The present study illustrates how formative teacher evaluation can be a meaningful tool in 
teacher improvement. It shows how formative evaluation can help teachers better analyze and 
understand teaching and take steps to improve their practice. It also provides support to the idea 
that teachers do want opportunities for evaluation and development when they feel these 
opportunities truly help them and are not forced upon them (see, Gitlin, 1989). In horizontal 
teacher evaluation, teachers seemed to find an evaluation program that fit their needs and was 
useful to them. They were able to use the program to address their specific teaching concerns and 
interests. The program was a tool to help them work together to better understand their teaching 
and improve their practice. As one teacher said in comparing this program to others in which she 
had participated, “I’m in awe that this is really helpful .... It matches reality and is practical.” 
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Appendix A 

Description of Coding Scheme 

All data for this study were coded according to eight general themes or categories: 
background, culture, goals, thoughts, actions, events, program, teachers. Descriptions of these 
categories appear below. Data within each of the first six category were analyzed to identify 
important subcategories. Relationships between subcategories and changes in subcategories over 
time were also examined. The general categories and subcategories were used to describe the 
program context and teachers who participated in the program. They were also used in describing 
teachers’ goals, thoughts, and actions during the course of the program. 

General Categories 

B Background — teachers' backgrounds including family, education, past experiences, and 

past likes and dislikes; any events or situations that occurred before the present year 

C Culture — culture of the school including general statements about what teachers, 

administrators, students, and parents are like; school atmosphere, feeling, interactions, 
procedures, policies 

G Goals — goal or desire of what teachers want or think should happen in teaching 

T Thoughts — teachers' thoughts, ideas, beliefs, or attitudes related to teaching; observers 

thoughts; thought processes in teaching including decision-making and thinking behind 
and during teaching 

A Actions — general action, pattern, or summary statement about teaching; teachers 

explaining what generally happens or observers explaining generally what they saw 

E Events — specific event or instance in relation to teaching where specific time, place, and 

individuals are involved 

P Program — aspects of the Horizontal Teacher Evaluation Program or comments that link 

the program to specific outcomes 

[name] Teachers — name of teachers making specific comments or displaying certain actions 
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